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ABSTRACT 

The  high  attrition  rate  in  Naval  Flight  Officer  (NFO) 
training  has  been  the  subject  of  much  concern  and  attention 
for  some  time.   Of  particular  concern  is  the  exceedingly  high 
attrition  rate  in  Radar  Intercept  Officer  (RIO)  training. 
The  present  prediction  system  is  based  on  a  linear  regression 
technique  of  academic  and  flight  variables.   This  thesis 
attempts  to  improve  on  the  current  prediction  methods  by  up- 
dating the  variables  used,  adding  variables  based  on  students 
personality  traits  and  exploring  the  possibility  of  employing 
the  logistic  function  as  the  prediction  technique. 
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I.   INTRODUCTION 

The  cost  of  fully  training  a  Naval  Aviator  or  Naval 
Flight  Officer  (NFO)  to  assume  his  operational  role  in  the 
fleet  represents  a  large  investment  of  the  Navy's  time  and 
money.   The  Navy  could  receive  the  optimal  return  from  this 
investment  if  it  could  select  and  train  only  the  number  of 
people  that  are  required  to  meet  fleet  operational  require- 
ments.  Ideally  this  means  everyone  selected  for  training 
eventually  becomes  a  designated  Naval  Aviator  of  NFO.   Zero 
attrition  is,  of  course,  impossible  to  attain  in  a  training 
program  of  this  size.   Efforts  to  reduce  attrition  are  gen- 
erally cost-effective.   Since  the  student  input  to  the 
aviation  training  program  is  so  large  a  small  decrease  in 
the  attrition  rate  reflects  a  large  savings  to  the  Navy. 

A  student  can  attrite  from  a  training  program  for  any 
of  several  reasons,  and  it  is  generally  agreed  that  there 
is  no  one  simple  causitive  factor--nor,  for  that  matter,  a 
quick  and  easy  solution.   Some  of  the  primary  reasons  for 
student  attritions  are: 

1)  Voluntary  withdrawal  or  drop  on  request  (DOR) . 
Students  DOR  for  a  number  of  reasons  many  of  which 
are  difficult  to  predict. 

2)  Not  physically  qualified  (NPQ) .   Due  to  the  strin- 
gent physical  requirements  for  entry  into  the  train- 
ing program  this  is  a  small  percentage  of  total 
attrites . 


3)  Transfer  to  another  training  program  (NFO  to  pilot 
or  pilot  to  NFO) .   Again  this  is  a  small  percentage 
of  the  total  attrition. 

4)  Academic  failure.   Unsatisfactory  performance  in 
academic  work. 

5)  Flight  failure.   Unsatisfactory  inflight  performance 

6)  Practical  work  failure.   Unsatisfactory  performance 
in  a  ground  simulator  or  training  device. 

Since  some  attrition  is  inevitable  in  a  large  training 
program  it  seem  logical  and  cost-effective  to  reduce  the 
attrition  by  selecting  and  training  only  those  students  that 
have  a  good  chance  of  successfully  completing  the  program. 
This  then  becomes  a  problem  of  predicting  success  or  per- 
formance on  a  job.   Good  prediction  is  essential  to  the 
Navy  since  ultimately  operational  readiness  and  national 
defense  depends  on  it.   The  use  of  periodic  prediction  dur- 
ing the  course  of  a  training  program  also  provides  important 
information  to  training  specialists  for  effective  guidance 
and  counseling  of  the  student  as  he  progresses. 

The  business  of  prediction  is  a  popular  one.   There  are 
numerous  studies  and  cases  in  the  literature  dealing  with 
prediction  of  success  or  prediction  of  performance.   Basi- 
cally, they  fall  into  the  two  categories  of  predicting  aca- 
demic success  or  predicting  job  success.   As  early  as  1947, 
Eysenck  estimated  there  were  more  than  a  thousand  studies 
recorded  in  educational  literature  pertaining  to  academic 
prediction.   Although  the  number  of  studies  abound  they 
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widely  differ  in  procedures.   Stein  (1963)  stated  that 
studies  differ  from  each  other  in  criteria,  procedures,  the 
types  of  students,  and  the  method  in  which  the  data  were 
reported. 

These  studies  revealed  different  types  of  factors  being 
examined  for  their  use  as  predictors.   The  most  common  of 
these  are  intellective  factors,  prior  scholastic  performance 
and  sociological  factors.   Intellective  factors  are  measured 
by  aptitude  or  intelligence  tests  such  as  the  Graduate  Record 
Examination.   Prior  scholastic  performance  in  the  form  of 
academic  grades  is  considered  by  many  to  be  the  best  predic- 
tor of  academic  success.   Research  into  sociological  factors 
indicates  they  have  some  effect  on  academic  success  but  it 
is  extremely  difficult  to  determine  the  extent  (Wilson,  1969) 

More  recently  there  has  been  an  increased  interest  in 
the  relationship  of  job  performance  and  the  personal  char- 
acteristics that  predict  it.   There  have  been  many  signifi- 
cant developments  in  the  area  of  performance  research  by 
both  industry  and  the  military.   As  early  as  1950  Mandell 
produced  significant  results  in  predicting  job  success  of 
engineers.   His  predictors  were  constructed  from  five  dif- 
ferent tests  covering  physics,  mathematics,  evaluation  of 
hypothesis,  visualization  and  reading. 

It  has  long  been  thought  that  personality  factors  con- 
tribute highly  to  job  performance.   Cattell  emphasized  the 
importance  of  personality-ability  traits  in  prediction  of 
job  performance  in  1957.   As  Sechrest  (1968)  points  out, 


personality  tests  are  of  interest  to  the  extent  that  they 
are  indicative  in  some  manner  of  the  way  in  which  the  indi- 
vidual will  respond  in  an  inferential  chain  which  leads  to 
some  manner  of  behavior  or  performance.   Although  person- 
ality traits  are  generally  acknowledged  to  be  important  to 
human  performance,  efforts  to  use  them  as  predictors  of  job 
performance  have  met  with  little  success.   After  analyzing 
various  tests  used  for  predicting  success  on  the  job, 
Ghiselli  and  Barthol  in  1953  and  Guion  and  Gottier  in  1965 
concluded  there  is  no  generalized  evidence  that  personality 
measures  can  be  recommended  as  good  or  practical  tools  for 
employee  selection.   The  most  that  was  said  for  them  is  that 
they  are  helpful  only  if  it  is  specifically  and  competently 
determined  for  a  specific  situation  and  for  a  specific  cri- 
terion within  that  situation.   Kelly  (1967)  and  Buch  and 
Haggard  (1968)  emphasized  the  need  for  complex  multivariate 
design  and  analysis  to  properly  interpret  the  results  of 
personality  tests  and  to  determine  how  and  to  what  extent 
the  predictor  variables  explain  or  predict  the  response  as 
measured  by  the  dependent  variable  or  criterion. 

The  military  has  been  quite  active  in  conducting  research 
on  predicting  job  performance.   Numerous  studies  conducted 
by  the  United  States  Air  Force  (Tupes,  19  59;  Lichenstein  and 
Hahn,  19  62;  Tupes,  Carp  and  Borg,  19  57;  Tupes,  19  63;  Judy, 
1962)  have  attempted  to  predict  job  performance  from  Officer 
Effectiveness  Reports  (OER) .   These  studies  have  addressed 
various  aspects  of  job  performance  resulting  in  varying 
degrees  of  success  in  prediction  of  performance. 
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The  United  States  Navy  has  done  considerable  research 
on  predicting  success  in  the  Naval  Aviation  Training  Program, 
Since  1963,  the  Aerospace  Psychology  Division  of  the  Naval 
Aerospace  Medical  Institute  has  provided  information  to 
naval  training  adminstrators  on  predicting  success  in  avia- 
tion training  (Shoenberger ,  Wherry  and  Berkshire,  1963). 
Upon  request,  administrators  are  given  the  computed  proba- 
bility based  on  multivariate  techniques  of  a  specific  stu- 
dent successfully  completing  the  flight  program.   These 
probabilities  are  obtained  by  appropriately  weighing  past 
performance  measures  such  as  initial  selection  test  scores, 
academic  course  grades  and  flight  grades.   Knowledge  of 
such  probabilities  has  improved  the  accuracy  of  decisions 
regarding  the  indentif ication  of  marginal  student  pilots. 
Such  data  could  be  used  to  increase  the  efficiency  of  uti- 
lization of  pilot  training  facilities  and  personnel. 

In  the  early  and  mid  1960 's  advent  of  multi-crew  air- 
craft such  as  the  P-3,  F-4,  A-5,  A-6  and  E-2  emphasized  the 
need  for  an  improved  training  program  for  the  Naval  Flight 
Officer  in  order  to  provide  the  skilled  personnel  to  operate 
these  new  highly  complex  and  diverse  aircraft.   Along  with 
the  expansion  of  the  NFO  training  program  came  a  need  for  a 
prediction  system  to  reduce  attrition  in  this  rapidly  ex- 
panding program  and  thereby  increase  the  efficiency  of  uti- 
lization of  the  available  training  assets.   A  prediction 
system  similar  to  the  one  in  existence  for  pilot  training 
was  developed  for  the  Naval  Flight  Officer  (Peterson,  Booth, 

Lane  and  Ambler,  1967). 
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Technological  development  in  naval  aviation  continued 
to  generate  more  intensive  specialization  in  NFO  training 
and  increased  the  number  of  advanced  training  specialties 
open  to  the  student  Naval  Flight  Officer.   Presently  there 
are  five  specialties  or  advanced  pipelines  available  to  the 
NFO.   Figure  1,  taken  from  the  Chief  of  Naval  Air  Training 
Instruction  1.500.4  of  March,  1975  is  a  diagram  of  student 
NFO  flow  from  the  initial  stages  of  training  to  the  various 
advanced  pipelines.   All  student  NFO's  receive  a  common 
core  of  training  up  through  the  NFO  basic  training  squadron, 
VT-10.   It  is  here  in  VT-10  that  the  student  states  his 
preferences  as  to  which  advanced  pipeline  he  would  prefer 
to  enter.   The  student  is  given  his  choice  whenever  possible, 
but  needs  of  the  service  and  availability  are  the  ultimate 
criteria  for  assignment. 

The  student  NOS ' s  selection  of  an  advanced  pipeline  is 
a  critical  point  in  his  training.   With  approximately  six 
months  of  training  completed  it  is  believed  that  those  stu- 
dents who  are  destined  to  attrite  would  have  done  so  by  this 
stage  of  training.   If  this  were  true  then  the  more  advanced 
and  expensive  training  would  be  relatively  free  of  attrition. 
However,  this  is  not  the  case.   Attrition  in  the  advanced 
pipelines  continue  to  take  a  costly  toll.   Statistics  held 
by  the  Office  of  the  Chief  of  Naval  Education  and  Training 
show  an  attrition  rate  of  about  ten  percent  for  the  student 
NFO  after  he  leaves  basic  training.   For  the  RIO  program 
alone  the  figure  is  about  twenty-five  percent. 


12 


COIvih/iON    CORE 


Environmental 
Indoctrination 

Basic  (VT-10) 


VP 

VQ 

A5 

F4 

VQ 

VAQ 

A6 

F1  4 

VX 

A3 

VS 

S3 

Figure  1 
Diagram  of  SNFO  F  low 


13 


A  methodology  was  developed  by  the  Aerospace  Psychology 
Division  of  the  Naval  Aerospace  Medical  Research  Laboratory 
to  assist  in  the  problem  of  assigning  trainees  to  one  among 
several  specialities  (Ambler,  Rickus  and  Booth,  1970).   This 
is  a  personnel  decision  method  that  alters  the  traditional 
concept  of  "assignment"  to  the  concept  of  "prevention  of 
misassignment . "   Multiple  regression  analysis  is  applied  to 
the  quantitative  variables  from  initial  screening  and  basic 
training  for  a  sample  of  students  from  each  type  of  advanced 
training.   The  dichotomous  criteria  of  pass  versus  fail  is 
used  to  develop  prediction  equations  for  each  pipeline. 
These  equations  were  then  incorporated  into  a  system  that 
permits  immediate  feedback  information  to  personnel  and 
training  officers  regarding  whether  a  particular  student 
under  consideration  for  assignment  to  an  advanced  pipeline 
would  have  a  high  or  low  probability  of  completing  the  ad- 
vanced training. 

Although  these  efforts  at  prediction  in  the  NFO  program 
met  with  some  success  it  did  not  enjoy  the  same  degree  of 
success  as  was  gained  with  the  pilot  prediction  system.   Of 
particular  concern  to  the  training  administrators  was  the 
exceedingly  high  attrition  rate  of  the  Radar  Intercept  Offi- 
cer (RIO)  student,  those  training  to  fly  in  the  rear  cockpit 
of  the  F-4  aircraft.   Ambler,  Rickus  and  Booth  (1970)  en- 
countered a  RIO  attrition  rate  of  thirty-five  percent  for  the 
sample  studied. 


14 


Although  the  present  prediction  system  contains  only 
variables  from  the  selection  test  scores  and  academic  and 
flight  grades  there  has  been  an  ongoing  effort  to  find  other 
types  of  variables  for  inclusion  in  the  selection  and  predic- 
tion procedure.   In  particular,  there  has  been  several  ef- 
forts to  include  personality  traits  in  the  prediction  equa- 
tions.  The  military  had  enjoyed  considerable  success  com- 
pared with  the  civilian  world  in  the  use  of  personality  in- 
ventories for  selection  and  screening  tools  to  predict  the 
problem  of  adjustment  to  military  life  (Ellis  and  Conrad, 
1948).   In  addition,  there  was  an  expectation  based  on  logic 
that  personality  is  related  to  success  in  flight  training, 
however,  efforts  to  relate  personality  measurements  to  this 
criteria  have  been  mostly  unsuccessful.   Past  research  re- 
veals attempts  to  use  different  personality  instruments  as 
they  are  created  in  the  literature  in  hopes  of  finding  one 
useful  as  a  prediction  device.   Schoenberger  (1963)  used 
the  Bass  SIT  inventory  to  obtain  a  correlation  between  per- 
sonality orientation  and  success  in  aviation  training.   He 
concluded  that  there  was  very  little  relationship  between 
the  two.   In  1965  the  Edwards  Personal  Preference  Schedule 
(EPPS)  was  evaluated  as  a  predictor  of  success  in  naval 
flight  training  (Peterson,  Lane  and  Kennedy,  1965)  .   The 
EPPS  failed  to  discriminate  between  those  who  successfully 
completed  training  and  those  who  failed.   In  1966  a  study 
investigated  five  different  personality  scales  for  possible 
use  as  predictors  (Fleischman,  Ambler,  Peterson  and  Lane, 
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1966).   The  five  scales  were:  (a)  Cattell's  Sixteen  Person- 
ality Questionnaire,  (b)  the  Taylor  Manifest  Anxiety  Scale, 
(c)  the  Alternate  Manifest  Anxiety  Scale,  (d)  the  Pensacola 
Z  Scale,  and  (e)  the  Adjective  Check-List.   The  results  were 
considered  promising  but  no  attempt  was  made  to  cross-vali- 
date the  results  and  no  further  attempts  were  made  to  develop 
these  scales  as  predictors.   Another  study  attempted  to  uti- 
lize the  Objectively  Scoreable  Apperception  Test  (OAT)  as  a 
selection  and  prediction  device  (Beal  and  Waldeisen,  1969). 
It  was  concluded  that  the  OAT  as  it  currently  existed  was 
not  suitable  for  inclusion  as  a  primary  predictor. 

The  primary  reasons  for  student  attrition  were  listed 
earlier  in  this  paper.   Most  attrition  is  caused  by  either 
voluntary  withdrawals  (DOR)  or  failures  in  academic  or 
flight/practical  work  performance.   Most  of  the  DOR's  occur 
because  of  dissatisfaction  with  the  system  or  training  pro- 
gram.  This  dissatisfaction  could  arise  during  training  as 
a  result  of  changes  in  syllabi,  disruptions  in  the  training 
flow  or  objection  with  advanced  pipeline  selection.   The 
root  of  the  student's  dissatisfaction  could  also  have  started 
prior  to  training  in  the  recruiting  area.   Perhaps  the  stu- 
dent was  seeking  entrance  to  pilot  training  but  physical 
problems  resulted  in  his  taking  NFO  training  as  a  trial  al- 
ternative.  Or  perhaps  the  recruit  was  not  fully  appraised 
of  the  nature  of  the  training  and  the  career  for  which  he 
was  applying. 
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Because  there  are  potentially  many  diverse  reasons  and 
circumstances  that  may  contribute  to  a  student's  decision 
to  withdraw  voluntarily  from  the  program  it  is  reasonable 
to  expect  that  the  DOR  type  attrition  would  be  difficult  to 
predict.   A  more  fruitful  area  for  reducing  attrition  seems 
to  be  in  prediction  of  those  who  will  attrite  through  aca- 
demic or  flight/practical  work  failure.   To  this  end  the 
purpose  of  this  thesis  was  to  investigate  ways  to  improve 
the  method  of  predicting  success  by: 

1)  Updating  the  existing  prediction  equations  now  being 
used.   Using  the  present  methodology  of  linear  re- 
gression the  most  current  academic  and  flight  data 
was  used  in  hopes  of  improving  predictability. 

2)  Including  another  personality  measurement  device  to 
investigate  the  possibility  of  increasing  the  cor- 
relation of  success  or  failure  with  personality 
traits . 

3)  Apply  a  different  mathematical  model  for  deriving 
the  predictor  equations. 

This  paper  will  deal  only  with  the  RIO  pipeline  since  it 
is  the  area  of  highest  attrition.   However/  the  analysis 
techniques  could  be  applied  to  any  of  the  advanced  pipeline 
areas . 
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II.   METHOD 

For  the  first  two  purposes  of  this  paper,  updating  the 
existing  prediction  method  and  incorporating  personality 
variables,  a  multiple  linear  regression  analysis  was  used. 
The  BIOMED  series  02R  stepwise  regression  routine  (University 
of  California,  1973)  executed  on  a  IBM  360/67  computer  was 
utilized  for  the  computation.   For  the  third  purpose  of 
trying  a  different  model  for  analysis  a  logistic  transforma- 
tion was  applied  as  opposed  to  a  linear  transformation.   A 
comparison  of  the  two  transformation  techniques  will  be  fur- 
ther discussed  in  a  later  section  of  this  paper. 

A.   DATA 

The  academic  variables  entered  for  analysis  were  derived 
from  three  different  areas;  selection  tests,  Environmental 
Indoctrination  and  Basic  Training  (VT-10) .   The  twenty  vari- 
ables derived  and  their  sources  are  listed  in  Table  I.   The 
flight  and  practical  work  scores  were  obtained  from  the  VT-10 
flight  syllabus  and  the  1D23  simulator  syllabus  to  make  a 
total  of  twenty- two  academic  and  flight  variables.   The  per- 
sonality variables  were  obtained  from  scores  on  the  Omnibus 
Personality  Inventory  (OPI)  (Heist  and  Yonge) .   The  OPI  was 
developed  for  the  purpose  of  providing  a  meaningful,  differ- 
entiation description  of  students  and  a  means  of  assessing 
change  rather  than  a  device  or  instrument  for  testing  a 
specific  theory  of  personality.   The  fourteen  scales  that 
make  up  the  OPI  are  listed  in  Table  II. 
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TABLE  I 
LIST  OF  ACADEMIC  AND  FLIGHT  VARIABLES 

SELECTION  TESTS 

Aviation  Qualification  Test  (AQT) 
Mechanical  Compreshension  Test  (MCT) 
Biographical  Inventory  (BI) 
Spatial  Apperception  Test  (SAT) 
Flight  Aptitude  Rating  (FAR) 


ENVIRONMENTAL  INDOCTRINATION 


Aerodynamics  (AERO) 
Engineering  (ENG) 


BASIC  (VT-10) 

Visual  Navigation  (VN) 

Dead  Reckoning  (DR) 

Flight  Rules  and  Regulations  (FRR) 

Airways  Navigation  (AN) 

Instrument  Ground  School  (IGS) 

Basic  Meteorology  (BMT) 

Advanced  Meteorology  (AMT) 

Flight  Support  (FS) 

Electricity  and  Electronics  (EE) 

Radar  Systems  (RS) 

Computer  Systems  (CS) 

Electronic  Warfare  (EW) 

Advanced  Systems  (AS) 


FLIGHT/SIMULATOR 

VT-10  Flight  Sylabus  Grade  (FLT) 
VT-10  1D23  Simulator  Grade  (TRAN) 
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TABLE  II 
SCALES  OF  THE  OMNIBUS  PERSONALITY  INVENTORY 


Thinking  Introversion  (TI) 
Theoretical  Orientation  (TO) 
Estheticism  (Es) 
Complexity  (Co) 
Autonomy  (Au) 

Religious  Orientation  (RO) 
Social  Extroversion  (SE) 
Impulse  Expression  (IE) 
Personal  Integration  (PI) 
Anxiety  Level  (AL) 
Altruism  (Am) 
Practical  Outlook  (PO) 
Masculinity-Femininity  (MF) 
Response  Bias  (RB) 
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B.  SUBJECTS 

Academic  and  flight  data  on  160  students  was  used  for 
the  analysis.   These  students  were  Navy  or  Marine  officers 
in  the  Naval  Flight  Officer  Training  Program  who  completed 
VT-10  and  started  Radar  Intercept  Training  at  VT-86.   The 
group  consisted  of  those  RIO  selectees  starting  from  VT-10 
class  437  graduating  from  basic  in  May,  1974,  through  the 
December,  1975,  VT-86  RIO  graduating  class.   Twenty-eight 
of  these  160  students  attrited  in  the  RIO  phase  for  academic/ 
flight  failure,  a  failure  rate  of  17.5  percent. 

OPI  data  was  available  on  only  93  of  the  original  160 
subjects.   Therefore,  analysis  involving  use  of  the  person- 
ality variables  was  restricted  to  these  93  subjects.   The 
OPI  test  was  administered  to  the  subjects  early  in  training 
by  the  staff  of  the  United  States  Medical  Research  Labora- 
tory, Naval  Air  Station,  Pensacola,  Florida. 

C.  PROCEDURE 

The  criterion  used  here  was  completion  of  RIO  training 
versus  attrition  by  reason  of  academic  or  flight/practical 
work  failure.   A  zero  was  assigned  as  the  dependent  variable 
if  the  student  attrited  and  a  one  was  assigned  if  the  stu- 
dent completed  the  program.   For  this  study  voluntary  with- 
drawals or  DOR's  were  eliminated  in  arriving  at  the  final 
sample  of  160  students.   First  the  group  as  a  whole  was 
analyzed  to  determine  the  highest  multiple  correlation  with 
the  criterion.   Then  the  subjects  were  randomly  divided  into 
two  groups.   Two- thirds  of  the  initial  group,  108  subjects 
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were  again  analyzed  in  the  same  manner  as  the  original  160 
to  obtain  predictor  variables,  their  weights  and  a  multiple 
R.   The  second  group  of  52  subjects  was  used  to  cross-vali- 
date the  results  obtained  from  the  first  group.   The  weights 
computed  from  the  first  group  were  applied  to  the  variables 
from  the  second  group  to  obtain  predicted  values  of  the 
criterion.   These  predicted  values  were  then  compared  to  the 
actual  dependent  variable  to  calculate  a  correlation  between 
the  actual  and  predicted. 

The  linear  regression  analysis  was  done  first  using  only 
the  twenty  two  academic  and  flight  variables  (the  existing 
method  of  determining  predictor  scores) .   Then  the  regres- 
sion analysis  was  done  for  the  OPI  sample  using  only  the 
fourteen  personality  variables.   Using  the  results  of  the 
OPI  analysis,  the  three  most  heavily  weighted  personality 
variables  were  then  added  to  the  twenty-two  academic  and 
flight  variables  for  similar  analysis. 

After  the  analysis  was  completed  on  the  total  group, 
the  academic  and  flight  variable  regression  analysis  was 
then  applied  to  the  Navy  and  Marine  students  as  separate 
groups.   This  was  attempted  to  investigate  the  possibility 
that  the  predictor  equations  and  results  are  significantly 
different  when  applied  to  the  total  group  as  opposed  to 
separate  Navy  and  Marine  groups.   The  existing  prediction 
system  treats  all  the  students  as  one  group. 

For  the  logistic  transformation  analysis  the  computation 
was  performed  utilizing  only  the  academic  and  flight  variables 
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In  this  way  the  logistic  technique  can  be  compared  directly 
with  results  obtained  by  the  presently  used  linear  model. 
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III.   ANALYSIS  AND  RESULTS 

This  section  contains  a  discussion  of  the  analysis  tech- 
niques used  and  the  results  obtained  from  the  analysis.   The 
two  methods  of  computation  -  linear  regression  and  logistic 
transformation  will  be  discussed  separately. 

A.   LINEAR  REGRESSION  MODEL 

The  linear  regression  model  is  of  the  form: 

Y  =  BX  +  e 

where,    Y  is  a  vector  of  dependent  variables,  pass/fail 
(1/0) . 
B  is  a  vector  of  coefficients,  or  Beta  weights 
X  is  the  matrix  of  independent  variables, 

academic/flight  or  OPI  scores 
e  is  an  error  term 

In  performing  the  regression  analysis  certain  assumptions 
are  made  about  the  errors;  the  errors  are  independent,  have 
zero  mean,  constant  variance  and  are  normally  distributed 
(Draper  and  Smith,  1966).   After  running  the  regression  rou- 
tine with  the  academic  and  flight  variables  a  plot  of  the 
residuals  against  the  criterion  and  against  the  independent 
variables  revealed  an  error  in  analysis.   The  plot  indicated 
that  the  assumptions  had  been  violated  and  that  the  linear 
model  may  not  be  the  proper  model  to  use.   This  arises  from 
the  fact  that  the  dichotomous  criteria  of  pass/fail  is 
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actually  a  series  of  Bernoulli  trails  resulting  in  a  bino- 
mial variable  not  a  normal  one.   It  is  for  this  reason  the 
logistic  transformation  was  selected  as  an  alternative  meth- 
od of  analysis.   Advantages  of  the  logistic  method  will  be 
discussed  later  in  this  section.   Another  problem  arising 
when  linear  regression  is  used  on  a  dichotomous  criteria  is 
that  the  predicted  values  from  the  regression  equation  are 
not  limited  to  falling  within  the  unit  interval.   It  is 
possible  to  get  predictor  scores  that  are  less  than  zero  or 
greater  than  one.   For  this  reason  the  predictor  scores  are 
not  directly  interpretable  as  probabilities  of  success  or 
failure.   The  sole  advantage  of  using  the  linear  regression 
technique  is  it  requires  relatively  simple  calculations  to 
arrive  at  the  beta  weights. 

The  means  and  standard  deviations  of  the  academic  and 
flight  variables  for  the  160  students  are  shown  in  Table 
III.   Table  IV  is  the  intercorrelation  matrix  including  all 
the  academic/flight  predictor  variables  and  the  pass/fail 
criterion.   When  all  academic/flight  variables  were  used  in 
the  stepwise  regression,  eight  were  significant  at  the  .05 
level.   However,  the  contribution  of  the  last  three  variables 
selected  at  this  level  was  not  considered  sufficient  to  war- 
rant their  inclusion  in  the  predictor  score  formula.   Thus, 
the  weights  to  be  applied  to  the  first  five  variables  chosen 
were  computed.   The  five  variables  chosen  and  their  multiple 
R  are  listed  in  Table  V.   As  mentioned  earlier,  cross-vali- 
dation was  accomplished  by  dividing  the  sample  randomly  and 
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TABLE  III 

MEANS  AND  STANDARD  DEVIATIONS 
OF  ACADEMIC  AND  FLIGHT  VARIABLES 


Variable  Mean  Std.  Dev, 

Pass/Fail  .82  0.38 

AQT  5.49  1.51 

FAR  5.71  2.21 

MCT  10.34  3.36 

SAT  11.42  3.48 

BI  10.81  3.96 

ENG  49.81  9.41 

AERO  50.9  6  9.3  7 

VN  51.41  9.70 

DR  50.26  8.08 

FRR  49.91  10.67 

AN  51.91  9.07 

IGS  51.54  9.08 

EE  54.57  9.06 

RS  50.11  8.43 

CS  51.36  9.25 

EW  50.29  9.08 

AS  50.84  8.49 

BMT  51.87  9.4  3 

AMT  49.8  8  9.16 

FS  53.46  9.51 

FLT  3.05  0.06 

TRAN  3.06  0.13 
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TABLE  IV 

INTERCORRELATION  MATRIX  OF 
ACADEMIC  AND  FLIGHT  VARIABLES  AND  THE  CRITERION 


P/F  AQT  FRR  MCT  SAT   BI  ENG  AER   VN  DR  FRR  AN  IGS 

P/F   1.0  .19  .14  .17  .07  .17  .22  .34  .24  .38  .05  .24  .17 

AQT  1.0  .36  .37  .29  .18  .41  .38  .46  .33  .18  .35  .24 

FAR  1.0  .74  .67  .78  .37  .34  .57  .34  .20  .26  .19 

MCT  1.0  .32  .47  .38  .34  .41  .30  .17  .24  .12 

SAT  1.0  .25  .16  .11  .42  .25-. 01  .15  .11 

BI  1.0  .34  .35  .41  .23  .22  .20  .22 

ENG  1.0  .51  .52  .39  .32  .49  .34 

AER  1.0  .45  .35  .19  .44  .36 

VN  1.0  .55  .28  .46  .32 

DR  1.0  .16  .34  .15 

FRR  1.0  .45  .32 

AN  1.0.40 

IGS  1.0 


EE   RS   CS   EW   AS  BMT  AMT   FS  FLT  TRAN 


P/F 

.34 

.21 

.22 

.08 

.19 

.21 

.08 

.23 

.34 

.40 

AQT 

.44 

.46 

.37 

.20 

.26 

.27 

.03 

.17 

.25 

.34 

FAR 

.28 

.36 

.29 

.06 

.26 

.26 

.14 

.10 

.33 

.33 

MCT 

.27 

.36 

.29 

.06 

.23 

.17 

.07 

.03 

.20 

.29 

SAT 

.21 

.20 

.25- 

-.01 

.18 

.11 

.05 

.09 

.27 

.16 

BI 

.22 

.30 

.17 

.12 

.21 

.29 

.19 

.14 

.23 

.26 

ENG 

.44 

.60 

.36 

.26 

.31 

.31 

.21 

.38 

.47 

.44 

AER 

.44 

.58 

.48 

.25 

.43 

.42 

.24 

.28 

.31 

.48 

VN 

.48 

.54 

.46 

.22 

.42 

.47 

.31 

.30 

.42 

.49 

DR 

.33 

.36 

.31 

.21 

.32 

.23 

.25 

.27 

.44 

.45 

FRR 

.16 

.35 

.10 

.20 

.34 

.26 

.35 

.26 

.38 

.29 

AN 

.44 

.54 

.38 

.27 

.43 

.34 

.28 

.43 

.50 

.52 

IGS 

.38 

.42 

.33 

.27 

.36 

.36 

.37 

.40 

.45 

.35 

EE 

1.0 

.54 

.43 

.31 

.38 

.32 

.29 

.31 

.35 

.38 

RS 

1.0 

.47 

.32 

.48 

.39 

.26 

.38 

.40 

.41 

CS 

1.0 

.23 

.43 

.29 

.17 

.22 

.25 

.29 

EW 

1.0 

.24 

.08 

.32 

.19 

.20 

.13 

AS 

1.0 

.30 

.28 

.32 

.26 

.33 

BMT 

1.0 

.32 

.33 

.28 

.32 

AMT 

1.0 

.32 

.34 

.20 

FS 

1.0 

.39 

.37 

FLT 

1.0 

.52 

TRAN 

1.0 
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TABLE  V 

FIVE  MOST  SIGNIFICANT  ACADEMIC/FLIGHT 
VARIABLES  FROM  LINEAR  MODEL 


* 
Variables  Cumulative  Multiple  R 


Simulator  (TRAN)  .396 

Dead  Reckoning  Nav  (DR)  .457 

Electricity  &  Electronics  (EE)             .487 

Visual  Navigation  (VN)  .501 

Aerodynamics  (AERO)  .512 


* 

All  variables  significant  at  .05  level 
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applying  the  stepwise  regression  method  to  each  sample. 
Cross-validation  resulted  in  essentially  the  same  variable 
weights  and  multiple  correlation  coefficients  for  the  sub- 
sample.   A  correlation  coefficient  of  .52  was  computed  on 
the  cross-validation  sub-sample. 

The  personality  variables  were  then  entered  in  the  step- 
wise regression  routine.   Table  VI  shows  the  means  and  stand- 
ard deviations  of  the  fourteen  personality  variables.   Table 
VII  is  the  intercorrelation  matrix  of  OPI  variables.   Only 
three  of  the  OPI  variables  were  significant  at  the  .05  level. 
The  OPI  variables  and  their  multiple  R  are  listed  in  Table 
VIII.   The  three  OPI  variables  were  then  added  to  the  twenty- 
two  academic/flight  variables  and  all  were  entered  into  the 
regression  routine.   Eight  variables  were  significant  at  the 
.05  level,  two  of  the  eight  were  personality  variables. 
Table  IX  lists  these  eight  variables  with  their  multiple  R. 
Although  all  eight  variables  appeared  to  increase  the  multi- 
ple R  enough  to  be  included  in  the  predictor  equation,  for 
simplicity  and  consistency  with  before  only  the  first  five 
were  used  for  cross-validation  purposes.   Table  X  shows  the 
results  when  the  academic/flight  variables  by  themselves  and 
the  OPI  variables  by  themselves  were  applied  to  the  Navy  and 
Marines  as  separate  groups.   Table  XI  presents  a  summary  of 
the  multiple  R  and  the  correlations  obtained  in  the  different 
analysis . 

B.   LOGISTIC  MODEL 

The  logistic  model  (Solberg,  Brown  and  Rutemiller,  1975) 
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TABLE  VI 

MEANS  AND  STANDARD  DEVIATIONS 
OF  OPI  VARIABLES 


Variables 


Mean 

Std.  Dev 

46.45 

8.19 

53.51 

9.26 

45.13 

8.97 

47.96 

9.12 

54.16 

7.40 

54.85 

8.26 

50.73 

10.18 

54.16 

9.41 

61.20 

7.59 

58.52 

6.61 

50.65 

9.49 

50.49 

7.58 

58.02 

5.98 

56.43 

9.50 

Thinking  Introversion  (TI) 
Theoretical  Orientation  (TO! 
Estheticism  (Es) 
Complexity  (Co) 
Autonomy  (Au) 

Religious  Orientation  (RO) 
Social  Extroversion  (SE) 
Impulse  Expression  (IE) 
Personal  Integration  (PI) 
Anxiety  Level  (AL) 
Altruism  (Am) 
Practical  Outlook  (PO) 
Masculinity-Femininity  (MF) 
Response  Bias  (RB) 
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TABLE  VIII 

OMNIBUS  PERSONALITY  INVENTORY  VARIABLES 
AND  MULTIPLE  R  FROM  LINEAR  MODEL 


Variable 


Cumulative  Multiple  R 


TI 
Am 
MF 
Es 
SE 
IE 
PO 
TO 
PI 
RB 
Au 
AL 
RO 
Co 


.194. 

.298^ 

.343 

.353 

.365 

.369 

.373 

.376 

.379 

.385 

.387 

.388 

.389 

.389 


Significant  at  .05  level 
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TABLE  IX 

EIGHT  MOST  SIGNIFICANT  VARIABLES 
OPI ,  ACAD/FLIGHT  AND  MULTIPLE  R 


Variable  Cumulutive  Multiple  R 


EE  .434 

TI  .524 

TRAN  .57  7 

MF  .599 

EW  .615 

AQT  .6  30 

FAR  .648 

BI  .665 


* 


All  variables  significant  at  .05  level 
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TABLE  X 

SUMMARY  TABLE 
NAVY  AND  MARINE  AS  SEPARATE  GROUPS 


Group  No.  of  Subjects  Cum.  Mult.  R, 

Navy  Acad/Flt  89  .624 

Navy  OPI  28  .607 

Marines  Acad/Flt  71  .729 

Marines  OPI  65  .448 
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TABLE  XI 
SUMMARY  TABLE  OF  MULTIPLE  R 


No 
Sub 

.  of 
jects 

All 
Vbls 

Total 
Acad/Flt 

160 

.58 

Navy 
Acad/Flt 

89 

.62 

Marines 
Acad/Flt 

71 

.73 

Total  OPI 

93 

.39 

Navy  OPI 

28 

.61 

Marine  OPI 

66 

.45 

Total 
Acad/Flt/OPI 

93 

.72 

Vbls  Sig     Five  Most 
at  .05      Sig  Vbls 


Cross  Validation 
Acad/Flt/OPI 

Cross  Validation 
Acad/Flt 

Cross  Validation 
Acad/Flt  Logistic 
Method 


54 


34 


67 


59 


52 


67 


.51 

.54 

.58 
.36 
.51 
.41 

.61 
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is  of  the  form 


Y  .  . Exp(BX) 

1  +  Exp(BX) 


where  Y,  B  and  X  are  the  same  as  in  the  linear  model.   The 
beta  weights  or  coefficients  are  obtained  by  maximum  likli- 
hood  estimation  (MLE) .   Professor  Gerald  Brown  of  the  Naval 
Postgraduate  School  has  written  a  computer  routine  that  com- 
putes the  beta  weights  for  the  logistic  model  in  a  manner 
similar  to  the  stepwise  multiple  linear  regression  routine. 
Variables  are  entered  or  removed  from  the  predictor  equations 
depending  on  the  significance  of  their  contribution  to  the 
dependent  variable.   The  primary  advantage  of  this  model  is 
that  the  dependent  variable  is  bounded  by  zero  and  one  and 
thus  can  be  directly  interpreted  as  a  probability  of  success 
or  failure.   A  Chi-Square  goodness  of  fit  test  is  utilized 
to  determine  the  significance  of  the  predictors  with  the 
pass/fail  criterion.   The  logistic  routine  resulted  in  the 
same  five  variables  weighting  the  heaviest  as  was  true  for 
the  linear  analysis.   Cross-validation  was  accomplished  on 
the  same  subset  of  52  subjects  as  the  linear  model.   The 
correlation  coefficient  obtained  on  cross-validation  was 
.68.   Table  XII  presents  a  summary  of  the  regression  coef- 
ficients obtained  by  the  two  different  analyses  on  the  dif- 
ferent subsets  of  data. 
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TABLE  XII 

REGRESSION  COEFFICIENTS 
FIRST  FIVE  VARIABLES 


Acad/Flt/OPI 

Acad/Flt 

Acad/Flt 

Linear  Model 

Linear  Model 

Logistic  Model 

Intercept 

-3.604 

-2.127 

-29.17 

TRAN 

.723 

.638 

6.627 

EE 

.018 

.008 

.814 

DR 

.126 

.141 

VN 

-.007 

-.075 

AERO 

.005 

.074 

EW 

-.006 

TI 

.017 

MF 

.012 

Model 

Y  = 

BX 

BX 

Exp(BX) 

l+Exp(BX) 
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IV .   CONCLUSIONS  AND  RECOMMENDATIONS 

The  variables  chosen  by  the  two  analysis  techniques 
(when  applied  to  the  total  group)  appear  to  have  face  valid- 
ity based  on  logic.   It  seems  logical  that  scores  received 
on  a  simulator  and  in  a  navigation  course  are  predictive  of 
future  performance  in  an  advanced  program  heavily  loaded 
with  cockpit  time  requiring  skills  acquired  from  basic  navi- 
gation.  It  is  encouraging  to  see  the  same  five  variables 
weighted  the  heaviest  in  both  the  linear  and  the  logistic 
analysis.   A  discouraging  result,  however,  is  the  poor  con- 
tribution of  the  selection  test  variables.   Although  they 
may  be  of  some  use  in  the  early  stages  of  training  they 
appear  to  have  little  predictive  validity  at  the  advanced 
stage.   One  interesting  result  of  the  analysis  is  the  sig- 
nificant increase  in  the  multiple  R  obtained  when  the  Navy 
and  Marine  students  are  treated  separately  instead  of  as 
one  group.   No  attempt  will  be  made  here  to  explain  this 
result  but  further  study  with  the  two  separate  groups  is 
needed  to  see  if  the  results  obtained  here  are  a  statistical 
artifact  or  if  the  results  are  in  general  valid  with  other 
Navy  and  Marine  groups.   It  is  encouraging,  however,  that 
the  two  groups  do  have  two  common  predictors — dead  reckoning 
navigation  and  simulator  grade. 

The  results  obtained  after  inclusion  of  the  personality 
variables  are  particularly  encouraging.   Their  contribution 
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to  the  multiple  R  is  greater  than  has  been  experienced  in 
past  studies.   The  fact  that  two  of  the  five  most  heavily 
weighted  variables  are  personality  traits  could  be  an  indi- 
cation of  the  importance  of  personality  factors  in  perform- 
ance.  Perhaps  the  Omnibus  Personality  Inventory  is  the 
long  sought  after  measure  of  personality  that  will  lead  to 
some  improved  prediction  ability. 

Encouraging  as  some  of  these  results  might  be,  it  must 
be  remembered  that  this  analysis  was  performed  under  the 
handicap  of  a  small  sample  size.   More  work  is  needed  with 
larger  samples  and  in  other  areas  throughout  the  training 
pipelines  to  confirm  the  trends  encountered  in  this  study. 
Although  the  personality  variables  were  not  included  in  the 
logistic  analysis  one  might  expect  the  addition  of  these 
variables  to  improve  the  predictive  ability  as  they  did  in 
the  linear  model. 

Further  analysis  needs  to  be  done  with  a  pass/DOR  and 
a  pass/attrite  criteria  in  addition  to  the  pass/fail  cri- 
teria investigated  in  this  study.   If  the  findings  of  this 
paper  are  an  indication  of  potential  results  and  if  the 
reported  technique  were  applied  to  other  stages  of  the 
training  program,  it  could  result  in  significant  improve- 
ment in  the  prediction  of  student  performance.   Ultimately, 
the  contribution  of  such  techniques  could  result  in  a  large 
savings  in  naval  training  programs. 
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